Skip to content

Conversation

@theduke
Copy link
Contributor

@theduke theduke commented Oct 15, 2025

fix: Prevent filesystem merge for packages

Previously adding additional dependencies to a WASI environment would
lead to a recursive file system merge, which would clone all files in
memory, which is very expensive.

This was caused by a very unstructed creation and extension logic in the
WasiFs, which could take on a complicated form of <dyn Filesystem>.

There previosly were attempts to downcast and extend without merging,
but those would fail in common use cases.

This is fixed by adding a dedicated WasiFs::Overlay variant, which is a
dedicated representation for the common wasix package environment use
case. This allows to deterministically extend an existing overlay with
additional read-only mounts.

@theduke theduke force-pushed the run-758-wasix-running-a-package-with-use-leads-to-recursive branch 3 times, most recently from 93939e0 to d33ca98 Compare October 15, 2025 18:15
Copilot AI review requested due to automatic review settings November 30, 2025 19:58
@theduke theduke force-pushed the run-758-wasix-running-a-package-with-use-leads-to-recursive branch from b4ac5d3 to 63d0b61 Compare November 30, 2025 19:58
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors the virtual filesystem handling in wasix to prevent recursive merge fallback operations when mounting packages. The core improvement is the introduction of an Overlay variant to WasiFsRoot that enables direct merging of UnionFileSystem instances without falling back to expensive recursive filesystem copies.

  • Changed filesystem type handling from Box to Arc throughout the codebase for better sharing semantics
  • Introduced WasiFsRoot::Overlay variant to enable efficient filesystem merging using UnionFileSystem::merge() instead of the recursive merge_filesystems() fallback
  • Extracted RelativeOrAbsolutePathHack to its own module for better code organization

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
lib/wasix/src/state/builder.rs Updated set_fs() to accept impl Into<Arc<...>> instead of Box, added internal set_fs_root() method, removed Arc wrapper from sandbox_fs() since TmpFileSystem is Clone
lib/wasix/src/state/env.rs Added handling for new WasiFsRoot::Overlay variant when injecting package commands
lib/wasix/src/fs/mod.rs Refactored WasiFsRoot enum to include Overlay variant, updated conditional_union() to use UnionFileSystem::merge() for overlay case, changed default_fs_backing() return type to Arc
lib/wasix/src/fs/relative_path_hack.rs New module containing extracted RelativeOrAbsolutePathHack implementation
lib/wasix/src/runtime/package_loader/load_package_tree.rs Changed filesystem() to return Option<UnionFileSystem> instead of boxed trait object, removed empty OverlayFileSystem creation, added webc version validation
lib/wasix/src/runners/wasi_common.rs Updated prepare_filesystem() to return WasiFsRoot and create Overlay variant, moved RelativeOrAbsolutePathHack to separate module
lib/wasix/src/runners/wasi.rs Updated to use duplicate() method when cloning filesystem
lib/wasix/src/runners/wcgi/runner.rs Updated to use duplicate() method when cloning filesystem
lib/wasix/src/bin_factory/binary_package.rs Changed webc_fs field from Arc<dyn FileSystem> to Option<Arc<UnionFileSystem>> for more specific typing
lib/virtual-fs/src/union_fs.rs Added merge() and duplicate() methods, made MountPoint cloneable
lib/virtual-fs/src/passthru_fs.rs Updated to use Arc instead of Box for inner filesystem
lib/c-api/src/wasm_c_api/wasi/mod.rs Added explicit type annotation for filesystem initialization
tests/lib/wast/src/wasi_wast.rs Updated test utilities to use Arc instead of Box for filesystem types

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Store the inner FS as an arc to make it more easily shareable.

Preserves constructor backwards compatibility.
Add the ability to merge a union fs into another using a given merge
mode.
@theduke theduke force-pushed the run-758-wasix-running-a-package-with-use-leads-to-recursive branch from ee8f735 to 55cda66 Compare November 30, 2025 20:36
@theduke theduke requested a review from syrusakbary as a code owner November 30, 2025 20:36
@theduke theduke force-pushed the run-758-wasix-running-a-package-with-use-leads-to-recursive branch from 55cda66 to 05cd3ab Compare November 30, 2025 20:38
@theduke theduke requested review from Arshia001 and Copilot November 30, 2025 20:38
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Previously adding additional dependencies to a WASI environment would
lead to a recursive file system merge, which would clone all files in
memory, which is very expensive.

This was caused by a very unstructed creation and extension logic in the
WasiFs, which could take on a complicated form of <dyn Filesystem>.

There previosly were attempts to downcast and extend without merging,
but those would fail in common use cases.

This is fixed by adding a dedicated WasiFs::Overlay variant, which is a
dedicated representation for the common wasix package environment use
case. This allows to deterministically extend an existing overlay with
additional read-only mounts.
@theduke theduke force-pushed the run-758-wasix-running-a-package-with-use-leads-to-recursive branch from 05cd3ab to f8ce5e3 Compare November 30, 2025 21:07
@Arshia001 Arshia001 merged commit b128c4c into main Dec 2, 2025
93 checks passed
@Arshia001 Arshia001 deleted the run-758-wasix-running-a-package-with-use-leads-to-recursive branch December 2, 2025 18:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants